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DETAILED ACTION 

Claim Rejections - 35 USC §112 

1 . The rejections presented in the Office Action dated 30 May 2006 under 35 USC § 1 12 
with respect to claim 22 have been withdrawn. 

2. The following is a quotation of the first paragraph of 35 U.S.C. 112: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

3. Claims 1-7, 9-18, 25-29, and 31-34 are rejected under 35 U.S.C, 112, first paragraph, 
because the specification, while being enabling for software units, does not reasonably 
provide enablement for hardware units. The specification does not enable any person skilled 
in the art to which it pertains, or with which it is most nearly connected, to make the invention 
commensurate in scope with these claims. In lines 18-22, page 27 of the descriptive portion of 
the specification, the units are disclosed as being hardware or software or both: "The figure is a 
block diagram focusing attention on functions and the functions identified in the function blocks 
can be realized in various ways such as using only hardware or only software, or a combination 
of both." However, the specification does not enable one of ordinary skill in the art to make a 
grouping unit, rendering processing unit, and consolidation unit in hardware. 

Claim Rejections - 35 USC §101 

4. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or composition of matter, or 
any new and useful improvement thereof, may obtain a patent therefor, subject to the conditions and 
requirements of this title. 

5. Claims 21-24 are rejected under 35 U.S.C. 101 because the claimed invention is 
directed to non-statutory subject matter. 
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6. Claims 21-24 are directed to a "storage medium," where the storage medium is not 
disclosed as being "computer-readable." In contrast, a claimed computer-readable medium 
storing a computer program is a computer element which defines structural and function 
interrelationships between the computer program and the rest of the computer which permit the 
computer program's functionality to be realized, and is thus statutory. Therefore, claims 21-24 
are directed to nonstatutory functional descriptive material. Additionally, claims 21-24 do not 
exclude carrier waves, or signals encoded with computer program instructions, which are nothing 
but the physical characteristics of a form of energy, and as such, claims 21-24 do not exclude 
nonstatutory natural phenomena. 

7. To expedite a complete examination of the instant application, the claims rejected under 
35 U.S.C. 101 as non-statutory subject matter are further rejected as set forth below in 
anticipation of applicant amending the claims to place them within the four categories of 
invention. 

Claim Rejections - 35 USC § 103 

8. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

9. The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 
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2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness 
or nonobviousness. 

10. Claims 1-6 and 20-27 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent Application Pub. No. 2002/0015055 to Foran in view of U.S. Patent No. 
6,016,150 to Lengyel et al. 

1 1 . With regard to claim 1, Foran discloses "an image processing apparatus comprising: 

a. a rendering processing unit which derives a three-dimensional subspace which 
contains the three-dimensional objects belonging to the same group to be an independent 
rendering unit {paragraph [0040]: "First, the dimensions of the scene are bounded by a 
rectangular volume as shown in FIG 2. Next, this volume is decomposed into 
rectangular subvolumes as shown in FIG. 3 such that each subvolume includes a portion 
of the overall scene... The position of each vertex is compared with the positions of the 
planar sides of the subvolumes to determine the subvolume to which the graphics 
primitive should be assigned. "), and generates independent image data for each subspace 
{paragraph [0044]: "In FIG. 5, at a step 506, the viewing position is communicated to 
each GPU. At a step 508, each GPU renders the graphics data that has been allocated to 
it. "); and 

b. a consolidation unit which generates final output image data to be displayed by 
consolidating the image data generated for each subspace {paragraph [0045]: "At a step 
512, the outputs of the individuals GPUs are combined by blending. "; Figure 7), 

c. wherein the grouping unit groups three-dimensional objects into groups in such a 
manner that the three-dimensional subspaces, each of which contains at least one three- 
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dimensional object belonging to the same group, are allowed to spatially overlap one 
another {paragraph [0040]: 'Where vertices of a graphics primitive indicate that it spans 
a boundary between subvolumes, the graphics primitive will be assigned to each 
subvolume. ")• 

12. Foran does not use the language allowing subspaces to spatially overlap; however, one of 
ordinary skill in the art would recognize that this feature is present as Foran discloses in 
paragraph [0040] that primitives that "span the boundary between subvolumes are assigned to 
each subvolume" and "Where parts of a graphics primitive exceed the boundaries of a 
subvolume, the GPU will clip that graphics primitive to the boundaries of the subvolume." Since 
a primitive can be assigned more than one subspace and primitives assigned to a subspace 
exceed the boundaries of the subvolume, while Foran does not expressly disclose the subvolumes 
overlap, the subspaces, the "portion of the overall scene" assigned to each rendering unit, must 
spatially overlap. 

13. Foran does not expressly disclose "a grouping unit which selects rendering strategy 
according to characteristics of input three-dimensional objects and groups the three-dimensional 
objects into groups in such a manner that the three-dimensional objects to which the same 
rendering strategy is applied are grouped into the same group;" or "performing rendering 
processing individually on the subspace by applying the group by group different rendering 
strategy." 

14. Lengyel et al discloses "a grouping unit which selects rendering strategy according to 
characteristics of input three-dimensional objects {lines 4-7 of column 10: "Preferably, the scene 
should be factored to identify scene elements (or sets of scene elements) that can be rendered to 
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separate layers at different update rates and spatial resolution. "; lines 33-38 of column 10 
(emphasis added): "Geometry factoring should consider the followin g properties of objects and 
their motions : 1. Relative velocity 2. Perceptual distinctness 3. Ratio of clear to "touched 11 
pixels. ") and groups the three-dimensional objects into groups in such a manner that the three- 
dimensional objects to which the same rendering strategy is applied are grouped into the same 
group {line 66 of column 10 through line 2 of column 11: "Background elements should be 
blurred by using a lower sampling rate. The main actor requires more samples in space and 
time. In order to make such tradeoffs, perceptually distinct objects must be separated into 
layers. ");" or "performing rendering processing individually on the subspace by applying the 
group by group different rendering strategy" {lines 15-1 7 of column 6: "In a layered pipeline, the 
spatial resolution of each of the layers can differ from each other and from the resolution of the 
output image. "; lines 24-25 of column 6: "The temporal resolution of each of the layers can also 
vary from one another and from the display update rate. "). 

15. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a grouping unit and a group by group different rendering strategy disclosed 
by Lengyel et al in the system disclose by Foran. The motivation for doing so would have been 
to improve computation efficiency and the appearance of the resulting displayed image, as 
suggested by Lengyel et al in line 66 of column 1 0 through line 2 of column 1 1 . Therefore, it 
would have been obvious to combine Lengyel et al with Foran to obtain the invention specified 
in claim 1 . 

16. With regard to claim 2, Lengyel et al discloses a viewing frustum {lines 9-12 of column 
22: "As described above, the preprocessor clips characteristic bounding polyhedron to a viewing 
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frustum that extends beyond the screen boundary (the expanded sprite extent). ") and a coordinate 
transform unit and rendering unit {lines 25-27 of column 12: "The render er 100 independently 
renders the fully illuminated layer by rendering a fully illuminated scene from the perspective of 
the viewpoint. ") for transforming the three-dimensional object to screen space and rendering the 
image {lines 12-16 of column 15: "When creating a sprite image, we must consider a new 
transform in the standard pipeline in addition to the modeling, viewing, and projection 
transforms: a 2D transformation that transforms samples in sprite coordinates to screen 
coordinates. "; lines 25-28 of column 15: "FIG. 8 is a diagram illustrating an example of 
mapping an object 120 from modeling coordinates 121 to sprite coordinates 122, and then 
mapping the object 120 from sprite coordinates to the screen coordinates 123. "). 

17. The combination of Foran and Lengyel et al does not explicitly disclose "a quadrangular 
truncated pyramid." Official Notice is taken that both the concept and the advantages of 
providing quadrangular truncated pyramid for perspective transformation and rendering of three- 
dimensional objects are well known and expected in the art. It would have been obvious to have 
included a quadrangular truncated pyramid in Lengyel et al as quadrangular truncated pyramids 
are known to provide a boundary for the viewable space of a virtual scene that is analogous to 
the visible volume of space in real-world image formation processes. Therefore, it would have 
been obvious to one of ordinary skill in the art to obtain the invention recited in claim 2. 

1 8. Claim 3 is met by the combination of Foran and Lengyel et al, wherein Lengyel et al 
discloses "the grouping unit, based on motion characteristics of the three-dimensional objects 
{lines 33-38 of column 10 (emphasis added): "Geometry factoring should consider the following 
properties of objects and their motions ... "), selects the rendering strategy on whether motion 
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blurring processing is applied or not, and groups the three-dimensional objects to which the 
motion blurring processing is applied, into the same group" (lines 26-31 of column 16 (emphasis 
added): "For a linear motion blur effect the sprite sampling along one of the axes may be 
reduced to blur along that axis. The sprite rendering transformation should align one of the 
coordinate axes to the object's velocity vector by setting the bounding slab directions to the 
velocity vector and its perpendicular. "). 

1 9. Claim 4 recites limitations similar in scope to those of claim 3. Claim 4 is rejected with 
the rational of claim 3. 

20. Claim 5 is met by the combination of Foran and Lengyel et al, wherein Lengyel et al 
discloses "the grouping unit, based on information related to level of detail in rendering the 
three-dimensional objections, selects the rendering strategy in which multi-resolution rendering 
is applied at a resolution depending on the level of detail, and groups the three-dimensional 
objects to be rendered at the same resolution into the same group" (line 66 of column 10 through 
line 2 of column 11: "Background elements should be blurred by using a lower sampling rate. 

* * 

The main actor requires more samples in space and time. In order to make such tradeoffs, 
perceptually distinct objects must be separated into layers. "; lines 54-57 of column 10: "Second, 
layered rendering more optimally targets rendering resources. Less important layers can be 
rendered at a lower spatial and temporal resolution to conserve resources for important 
layers. "). 

21 . Claim 6 recites limitations similar in scope to those of claim 5. Claim 6 is rejected with 
the rational of claim 5. 
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22. With regard to claim 20, Foran discloses "an image processing method comprising 
grouping a plurality of three-dimensional objects into groups" {paragraph [0040]: "First, the 
dimensions of the scene are bounded by a rectangular volume as shown in FIG. 2. Next, this 

■ 

volume is decomposed into rectangular subvolumes as shown in FIG. 3 such that each 
subvolume includes a portion of the overall scene... The position of each vertex is compared with 
the positions of the planar sides of the subvolumes to determine the subvolume to which the 
graphics primitive should be assigned. ") "performing rendering processing individually on a 
subspace which contains at least on three dimensional object belonging to the same group" 
{paragraph [0044]: "In FIG. 5, at a step 506, the viewing position is communicated to each 
GPU. At a step 508, each GPU renders the graphics data that has been allocated to it. ") and 
generating final image data to be displayed by consolidating rendering data of each subspace 
{paragraph [0045]: "At a step 512, the outputs of the individuals GPUs are combined by 
blending. "; Figure 7), wherein three-dimensional subspace, each of which contains at least one 
three-dimensional object belonging to the same group, are allowed to spatially overlap one 
another {paragraph [0040]: "Where vertices of a graphics primitive indicate that it spans a 
boundary between subvolumes, the graphics primitive will be assigned to each subvolume. ") 

23. With regard to claim 20, Foran does not disclose grouping "in such a manner that the 
three-dimensional objects to which same rendering strategy is applied are grouped into the same 
group and "applying the group by group different rendering strategy." 

24. With regard to claim 20, Lengyel et al discloses "grouping the three-dimensional objects 
which exist in a display area into groups in such a manner that the three dimensional objects to 
which the same rendering strategy is applied are grouped into the same group (lines 33-38 of 



4 
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column JO: "Geometry factoring should consider the following properties of objects and their 
motions: J. Relative velocity 2. Perceptual distinctness 3. Ratio of clear to "touched" pixels. "; 
line 66 of column 10 through line 2 of column 11); performing rendering processing individually 
by subspace unit by applying the group by group different rendering strategy to generate image 
data for each subspace (lines 15-17 of column 6; lines 24-25 of column 6; lines 7-12 of column 
6). 

25. With regard to claim 21 , Foran discloses "a storage medium storing a computer program 
executable by a computer {paragraph [0040]; "In an embodiment where the invention is 
implemented using software, the software may be stored in a computer program product and 
loaded into computer system 800 using removable storage drive 814, hard drive 812 or 
communications interface 824."), the program comprising: 

d. deriving a subspace which contains the three-dimensional objects belonging to the 
same group to be an independent rendering unit (paragraph [0040]: "First, the 
dimensions of the scene are bounded by a rectangular volume as shown in FIG. 2. Next, 
this volume is decomposed into rectangular subvolumes as shown in FIG. 3 such that 
each subvolume includes a portion of the overall scene... The position of each vertex is 
compared with the positions of the planar sides of the subvolumes to determine the 
subvolume to which the graphics primitive should be assigned. "; paragraph [0044]: "In 
FIG. 5, at a step 506, the viewing position is communicated to each GPU. At a step 508, 
each GPU renders the graphics data that has been allocated to it. "); and 
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e. generating final image data to be displayed in the display area by consolidating 
the image data generated for each subspace {paragraph [0045] : "At a step 512, the 
outputs of the individuals GPUs are combined by blending. "; Figure 7) 

f. wherein the grouping groups three-dimensional objects into groups in such a 
manner that the three-dimensional subspaces, each of which contains at least one three- 
dimensional object belonging to the same group, are allowed to spatially overlap one 
another {paragraph [0040]: "Where vertices of a graphics primitive indicate that it spans 
a boundary between subvolumes, the graphics primitive will be assigned to each 
subvolume. "). 

26. Foran does not disclose "group by group different rendering strategy." Lengyel et al 
discloses "selecting a rendering strategy according to characteristics of the three-dimensional 
objects" {lines 4-7 of column 10: "Preferably, the scene should be factored to identify scene 
elements (or sets of scene elements) that can be rendered to separate layers at different update 
rates and spatial resolution. "); "grouping the three-dimensional objects which exist in a display 
area into groups in such a manner that the three dimensional objects to which the same rendering 
strategy is applied are grouped into the same group {lines 33-38 of column 10: "Geometry 
factoring should consider the following properties of objects and their motions: 1. Relative 
velocity 2. Perceptual distinctness 3. Ratio of clear to "touched" pixels. "; line 66 of column 10 
through line 2 of column 11); performing rendering processing individually by subspace unit by 
applying the group by group different rendering strategy to generate image data for each 
subspace {lines 15-17 of column 6\ lines 24-25 of column 6; lines 7-12 of column 6). 
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27. With regard to claims 20 and 21, at the time of the invention, it would have been obvious 
to a person of ordinary skill in the art to incorporate a grouping unit and a group by group 
different rendering strategy disclosed by Lengyel et al in the system disclose by Foran. The 
motivation for doing so would have been to improve computation efficiency and the appearance 
of the resulting displayed image, as suggested by Lengyel et al in line 66 of column 10 through 
line 2 of column 11. Therefore, it would have been obvious to combine Lengyel et al with Foran 
to obtain the invention specified in claims 20 and 21. 

28. The combination of Foran and Lengyel et al does not expressly disclose "reading array 
data of a plurality of three-dimensional objects." Official Notice is taken that both the concept 
and the advantages of providing an array for reading three-dimensional objects of a scene are 
well known and expected in the art. It would have been obvious to have included the operation of 
reading array data in the system disclosed by the combination of Foran and Lengyel et al as 
arrays are known to provide a random access data retrieval and a simple implementation. 
Therefore, it would have been obvious to one of ordinary skill in the art to obtain the invention 
recited in claim 21. 

29. With regard to claim 22, the combination of Foran and Lengyel et al discloses the 
limitations of parent claim 21 . Lengyel et al further discloses "calculating a position of each of 
the three-dimension objects in a viewpoint coordinate system" (line 65 of column 6 through line 
3 of column 7: "In FIG. 2, the input 32 to a layered pipeline includes the set of 3D objects in the 
scene, and the position of objects, light sources, and the viewpoint for the scene. In the context 
of animation, the position of objects, light sources and the viewpoint can be time-varying, in 
which case, the position of these scene elements are represented as functions of time. ") "and 
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determining information related to level of detail in rendering each of the three-dimensional 
objects based on a distance from the viewpoint, and wherein said grouping groups the three- 
dimensional objects which exist in the display area into the groups according to the information 
related to level of detail (line 66 of column 10 through line 2 of column 11: "Background 
elements should be blurred by using a lower sampling rate. The main actor requires more 
samples in space and time. In order to make such tradeoffs, perceptually distinct objects must be 
separated into layers. "; lines 54-57 of column 10: "Second, layered rendering more optimally 
targets rendering resources. Less important layers can be rendered at a lower spatial and 
temporal resolution to conserve resources for important layers. "). 

30. Claim 23 and 24 is met by the combination of Foran and Lengyel et al, wherein Foran 
discloses "the rendering processing are performed in such a manner that the rendering processing 
for each subspace is distributed to a plurality of rendering processing units" {paragraph [0044]: 
"In FIG. 5, at a step 506, the viewing position is communicated to each GPU. At a step 508, each 
GPU renders the graphics data that has been allocated to it. "; 701-724 shown in Figure 7). 

3 1 . Claim 25 is met by the combination of Foran and Lengyel et al, wherein Lengyel et al 
discloses "the rendering strategy is a rendering algorithm applied to the three-dimensional 
objects" (lines 2-6 of column 6: "FIG. 2 is a diagram depicting a layered graphics rendering 
pipeline 30. Like a traditional frame buffer approach, the input to the pipeline is a 3D scene 32 
describing the position and visual attributes of the graphical objects in the scene. "). 

32. Claim 26 is met by the combination of Foran and Lengyel et al, wherein Lengyel et al 
discloses "the rendering algorithm is a hidden surface removal algorithm" (lines 43-45 of column 
29 (emphasis added): "The tiler 236 performs scan-conversion, shading, texturing, hidden- 
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surface removal anti-aliasing, translucency, shadowing, and blending for multi-pass 
rendering. "). 

33. Claim 27 is met by the combination of Foran and Lengyel et al, wherein Lengyel et al 
discloses "wherein the rendering algorithm is a shading algorithm" {lines 43-45 of column 29 
(emphasis added): "The tiler 236 performs scan-conversion, shading, texturing, hidden-surface 
removal, anti-aliasing, translucency, shadowing, and blending for multi-pass rendering 1 ") 

34. Claims 9-14, 17, and 18 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over U.S. Patent Application Pub. No. 2002/0015055 to Foran in view of U.S. Patent No. 
6,016,150 to Lengyel et al in view of Kwan Liu Ma, James S. Painter, Charles D. Hansen, 
Michael F. Krogh, "A Data Distributed, Parallel Algorithm for Ray-Traced Volume 
Rendering," October 25, 1993, Proceedings of the 1993 Symposium on Parallel Rendering, 
p. 15-22 (Ma et al). 

35. With regard to claims 9 and 10, the combination of Foran and Lengyel et al shows the 
limitations of the parent claim 1, but does not disclose assigning to rendering units based on 
performance and complexity. Ma et al teaches "the rendering processing unit comprises a 
plurality of rendering units {first paragraph of section 4: "Currently, the data distributor runs as 
a single 'host' process that determines the partitions of the data set, reads the data set piece by 
piece from disk and distributes it to a set of 'node 'processes that perform the actual rendering 
and compositing"') and distributes the rendering processing to the plurality of the rendering units 
according to complexity level of the rendering processing by subspace unit" {third paragraph of 
section 6: "The data subdivision can be done unevenly, taking into account the predicted 
capacity on each machine to try to balance the load. "). 
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36. With regard to claims 1 1 and 12, the combination of Foran and Lengyel et al shows the 
limitations of parent claim 1, but does not disclose assigning to rendering units based on 
performance and complexity. Ma et al teaches "the rendering processing unit comprises a 
plurality of rendering units (shown in the rejection of claims 9 and 10) with different processing 
performance and assigns the rendering processing to the plurality of the rendering units, each of 
which has the appropriate processing performance corresponding to complexity level of the 
rendering processing by subspace unit" (third paragraph of section 6: "Alternatively, the data 
can be subdivided into larger number of equal sized subvolumes and the more capable machines 
can be assigned more than one subvolume. "). 

37. With regard to claims 13 and 14, the combination of Foran and Lengyel et al shows the 
limitations of parent claim 1, but does not disclose external networks. Ma et al teaches "a 
communication unit which receives image data rendered by subspace unit from an external 
distributed rendering processing device connected with the apparatus via a network" (first 
paragraph of section 3.3: "Each computer can perform ray tracing independently; that is, there 
is no data communication required during the sub volume rendering. "; fourth paragraph of 
section 3.4: "In our example, as shown in (b), Computer 1 keeps only the left half-image and 
sends its right half image to its immediate right sibling, which is Computer 2. "), and "wherein 
the consolidation unit consolidates the image data received from the external distributed 
rendering processing device with the image data generated by the rendering processing unit and 
generates the final output image data to be displayed" (first paragraph of section 3.4: "The final 
step of our algorithm is to merge ray segments and thus all partial images into the final total 
image. "). 
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38. With regard to claims 17 and 18, Foran discloses "an image processing apparatus 
comprising: 

g. a rendering processing unit which derives a three-dimensional subspace which 
contains the three-dimensional objects belonging to the same group to be an independent 
rendering unit {paragraph [0040]: "First, the dimensions of the scene are bounded by a 
rectangular volume as shown in FIG 2. Next, this volume is decomposed into 

* 

rectangular subvolumes as shown in FIG. 3 such that each subvolume includes a portion 
of the overall scene.,. The position of each vertex is compared with the positions of the 
planar sides of the subvolumes to determine the subvolume to which the graphics 
primitive should be assigned. "), and generates independent image data for each subspace 
(paragraph [0044]: "In FIG 5, at a step 506, the viewing position is communicated to 
each GPU. At a step 508, each GPU renders the graphics data that has been allocated to 
it. "); and 

h. a consolidation unit which generates final output image data to be displayed by 
consolidating the image data generated for each subspace {paragraph [0045]: "At a step 
512, the outputs of the individuals GPUs are combined by blending. "; Figure 7), 

i. wherein the grouping unit groups three-dimensional objects into groups in such a 
manner that the three-dimensional subspaces, each of which contains at least one three- 
dimensional object belonging to the same group, are allowed to spatially overlap one 
another {paragraph [0040]: "Where vertices of a graphics primitive indicate that it spans 
a boundary between subvolumes, the graphics primitive will be assigned to each 
subvolume. "). 
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39. With regard to claims 1 7 and 1 8, Foran does not use the language allowing subspaces to 

spatially overlap; however, one of ordinary skill in the art would recognize that this feature is 

present as Foran discloses in paragraph [0040] that primitives that "span the boundary between 

subvolumes are assigned to each subvolume" and "Where parts of a graphics primitive exceed 

the boundaries of a subvolume, the GPU will clip that graphics primitive to the boundaries of the 

« 

subvolume." Since a primitive can be assigned more than one subspace and primitives assigned 
to a subspace exceed the boundaries of the subvolume, while Foran does not expressly disclose 
the subvolumes overlap, the subspaces, the "portion of the overall scene" assigned to each 
rendering unit, must spatially overlap. 

40. With regard to claims 17 and 18, Foran does not expressly disclose "a grouping unit 
which selects rendering strategy according to characteristics of input three-dimensional objects 
and groups the three-dimensional objects into groups in such a manner that the three-dimensional 
objects to which the same rendering strategy is applied are grouped into the same group;" or 
"performing rendering processing individually on the subspace by applying the group by group 

* • 

different rendering strategy." 

41. With regard to claims 17 and 18, Lengyel et al discloses "a grouping unit which selects 
rendering strategy according to characteristics of input three-dimensional objects (lines 4-7 of 
column 10: "Preferably, the scene should be factored to identify scene elements (or sets of scene 
elements) that can be rendered to separate layers at different update rates and spatial 
resolution, "; lines 33-38 of column 10 (emphasis added): "Geometry factoring should consider 
the followin g properties of objects and their motions : 1. Relative velocity 2. Perceptual 
distinctness 3. Ratio of clear to "touched" pixels. ") and groups the three-dimensional objects 



Application/Control Number: 10/653,791 Page 18 

Art Unit: 2628 

into groups in such a manner that the three-dimensional objects to which the same rendering 
strategy is applied are grouped into the same group (line 66 of column 10 through line 2 of 
column 11: "Background elements should be blurred by using a lower sampling rate. The main 
actor requires more samples in space and time. In order to make such tradeoffs, perceptually 
distinct objects must be separated into layers. ");" and "performing rendering processing 
individually on the subspace by applying the group by group different rendering strategy" (lines 
15-17 of column 6: "In a layered pipeline, the spatial resolution of each of the layers can differ 
from each other and from the resolution of the output image. "; lines 24-25 of column 6: "The 
temporal resolution of each of the layers can also vary from one another and from the display 
update rate. "). 

42. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to incorporate a grouping unit and a group by group different rendering strategy disclosed 
by Lengyel et al in the system disclose by Foran. The motivation for doing so would have been 
to improve computation efficiency and the appearance of the resulting displayed image, as 
suggested by Lengyel et al in line 66 of column 10 through line 2 of column 1 1 . 

43. With regard to claims 17 and 1 8, Foran and Lengyel et al does not disclose a distributed 
system. Ma et al teaches "plurality of image processing apparatus for exchanging information 
with each other via a network and performing distributed rendering processing," as recited in 
claim 17, and "wherein a processing result by the function block which is not included in this 
apparatus is received from the other apparatus and utilized," as recited in claim 18 (fourth 
paragraph of section 3. 4: "In our example, as shown in (b), Computer 1 keeps only the left half- 
image and sends its right half image to its immediate right sibling; which is Computer 2. "; first 
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paragraph of section 4.2: "Thus, using a set of high performance work stations connected by an 
Ethernet, our goal is to set up a volume rendering facility... "). Furthermore, Ma et al teaches 
that grouping, the rendering processing and the consolidation are functionally distributed among 
the plurality of the image processing apparatus (first paragraph of section 3.3: "Each computer 
can perform ray tracing independently; that is, there is no data communication required during 
the sub volume rendering. "; first paragraph of section 4: "Currently, the data distributor runs 
as a single 'host 'process that determines the partitions of the data set, reads the data set piece 
by piece from disk and distributes it to a set of 'node 'processes that perform the actual 
rendering and compositing"). 

44. With regard to claims 9-14, 17, and 18, at the time of the invention, it would have been 
obvious to a person of ordinary skill in the art to distribute the grouping unit, rendering 
processing unit, and consolidation unit disclosed by Lengyel et al across a plurality of image 
processing apparatuses as taught by Ma et al. The motivation for doing so would have been to 
accelerate rendering as stated by Ma et al in the first paragraph of section 1 . Therefore, it would 
have been obvious to further modify the combination of Foran and Lengyel et al with Ma et al to 
obtain the invention specified in claims 9-14, 17, and 18. 

45. Claims 15 and 16 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent Application Pub. No. 2002/0015055 to Foran in view of U.S. Patent No. 
6,016,150 to Lengyel et al in view of Kwan Liu Ma, James S. Painter, Charles D. Hansen, 
Michael F. Krogh, "A Data Distributed, Parallel Algorithm for Ray-Traced Volume 
Rendering," October 25, 1993, Proceedings of the 1993 Symposium on Parallel Rendering, 
p. 15-22 (Ma et al) and in further view of Jerrell Watts, Stephen Taylor, "A Practical 
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Approach to Dynamic Load Balancing, 9 ' March 1998, IEEE Transactions on Parallel and 
Distributed Systems, v. 9 n. 3, p. 235-248 (Watts et al). 

46. With regard to claims 15 and 16, the combination of Foran, Lengyel et al and Ma et al 
shows the limitations of parent claims 13 and 14, but does not show rendering processing load 
assigned corresponding to network distance. Watts et al teaches a task is assigned to a plurality 
of the distributed devices, each of which has different network distance corresponding a task's 
transfer cost (fourth paragraph of section 5.2: "In the first case, a task's transfer cost was taken 
to be the change in the distance from the actual location of its data structures to its proposed 
new location: i.e. the transfer for task i was.., where dist is a function which gives the network 
distance between any two computers... "). One of ordinary skill in the art would recognize that 
the level of detail is directly related to the amount of data and the transfer cost of the object to be 
rendered from the statement by Ma et al in the second paragraph of section 6: "For example, a 
processor tracing through empty space will probably finish before another processor working on 
a dense section of the data." 

47. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the network distance based load balancing method taught by Watts et al to 
distribute the rendering tasks disclosed by the Lengyel et al and Ma et al system according to 
transfer cost. Ma et al suggests in the third paragraph of section 6, "Performance of the 
distributed workstation implementation could be further improved by better load balancing"; 
thus, the motivation for doing so would have been to improve the performance of the rendering 
system. Therefore, it would have been obvious to modify the Lengyel et al and Ma et al 
combination with Watts et al to obtain the invention specified in claims 15 and 16. 
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48. Claims 28 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
U.S. Patent Application Pub. No. 2002/0015055 to Foran in view of U.S. Patent No. 
6,016,150 to Lengyel et al in view of U.S. Patent No. 5,986,659 to Gallery et al. 

49. With regard to claims 28 and 29, Lengyel et al discloses "the grouping unit, based on 
information related to level of detail in rendering the three dimensional objects, selects the 
rendering strategy which defocus processing is applied, and groups the three-dimensional objects 
to which the defocus processing is applied" (line 66 of column 10 through line 2 of column 11: 
"Background elements should be blurred by using a lower sampling rate. The main actor 
requires more samples in space and time. In order to make such tradeoffs, perceptually distinct 
objects must be separated into layers. "). Lengyel et al does not disclose a "focus depth 
depending on the level of detail." Gallery et al discloses a "defocus processing is applied at a 
focus depth depending on the level of detail" (lines 10-16 of column 3: "Specifying a point of 
focus P (a depth value at the place in the image that the observer is looking at), in order to give 
the appearance of focus/de-focus it is assumed that for pixels in the image with depth close to 
that of the point of focus the image should be sharp, but, as the depth of a pixel gets further away 
from the depth of the point of focus .(whether nearer or closer to the position of the observer) 
then the image should become more blurred. "). 

50. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to perform defocus processing at a focus depth as taught by Gallery et al in the system and 
method disclosed by the combination of Foran and Lengyel et al. The motivation for doing so 
would have been to model the process of image formation of a physical camera, thereby 
improving the realism of the resulting images as suggested by Gallery et al "Synthetically 
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generated computer graphics images normally suffer from the deficiency of being uniformly 
sharp, i.e. all parts of the image appear to be in focus" (lines 10-12 of column 1). Therefore, it 
would have been obvious to combine Gallery et al with Foran and Lengyel et al to obtain the 
invention specified in claim 28 and 29. 

51 . Claims 19 and 33 rejected under 35 U.S.C. 103(a) as being unpatentable over U.S. 
Patent Application Pub. No. 2002/0015055 to Foran in view of U.S. Patent No. 5,867,166 to 
Myhrvold et al. 

52. With regard to claim 19, Foran discloses "an image processing method comprising 
dividing a space into three-dimensional subspaces which are allowed to spatially overlap one 
another {paragraph [0040]: n First, the dimensions of the scene are bounded by a rectangular 
volume as shown in FIG. 2. Next, this volume is decomposed into rectangular subvolumes as 
shown in FIG. 3 such that each subvolume includes a portion of the overall scene... The position 
of each vertex is compared with the positions of the planar sides of the subvolumes to determine 
the subvolume to which the graphics primitive should be assigned. "; paragraph [0040] : 'Where 
vertices of a graphics primitive indicate that it spans a boundary between subvolumes, the 
graphics primitive will be assigned to each subvolume. ") and performing rendering processing 
independently by subspace unit on at least one three-dimensional object in each of the subspaces 
(paragraph [0044]: "In FIG. 5, at a step 506, the viewing position is communicated to each 
GPU. At a step 508, each GPU renders the graphics data that has been allocated to it. "), and 
consolidating the rendering data of at least one three-dimensional object in each of the 
subspaces" {paragraph [0045]: "At a step 512, the outputs of the individuals GPUs are 
combined by blending. "; Figure 7). 
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53. Foran discloses, "image blending accounts for the spatial relationships between the 
viewing position and the rendered images so that objects that are more near the viewing position 
overwrite those that are more distant" (paragraph [0045]). Foran does not expressly disclose 
"generating rendering data having depth information on a pixel by pixel basis" and "evaluating a 
distance in depth direction on a pixel by pixel basis." 

54. With regard to claim 19, Myhrvold et al discloses "an image processing method 
comprising dividing a space into subspaces (lines 18-24 of column 9: "A scene, or portions of a 
scene, can be divided into pixel regions (32x32 pixels in one specific implementation), called 
chunks. In one implementation, the system divides the geometry assigned to gsprites into chunks, 
but an alternative implementation could perform chunking without gsprites. The geometry is 
presorted into bins based on which chunk the geometry will be rendered into. ") generating 
rendering data having depth information on a pixel by pixel basis (lines 37-41 of column 67: 
"...and Z is the Z-value which represents the depth of the pixel from the eye point, Mis a 4x4 
pixel coverage bitmask for each pixeLwhich is partially covered... "), and consolidating the 
rendering data of the three dimensional object in each of the subspaces by evaluating a distance 
in depth direction on a pixel by pixel basis" (lines 12-16 of column 66: "To perform hidden 
surface removal, the pixel engine 406 compares depth values for incoming pixels (fully covered 
pixels or pixel fragments) with pixels at corresponding locations in the pixel or fragment 
buffers"). 

55. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the per-pixel depth calculations of Myhrvold et al in Foran. The motivation for 
doing so would have been to accurately determine visible surfaces, as suggested in Myhrvold et 
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al in lines 37-41 of column 67. Therefore, it would have been obvious to combine Myhrvold et al 
with Foran to obtain the invention specified in claim 19. 

56. With regard to claim 33, Foran discloses "an image processing apparatus comprising: 
j. a grouping unit which groups input three-dimensional objects into groups 
{paragraph [0040]: "First, the dimensions of the scene are bounded by a rectangular 
volume as shown in FIG. 2. Next, this volume is decomposed into rectangular subvolumes 
as shown in FIG. 3 such that each subvolume includes a portion of the overall 

scene... The position of each vertex is compared with the positions of the planar sides of 
the subvolumes to determine the subvolume to which the graphics primitive should be 
assigned. "); 

k. a rendering processing unit which derives a subspace which contains at least one 
three-dimensional object belonging to the same group to be an independent rendering unit 
and performs rendering processing individually on the subspace, and generates 
independent image data {paragraph [0044]: "In FIG. 5, at a step 506, the viewing 
position is communicated to each GPU. At a step 508, each GPU renders the graphics 
data that has been allocated to it. "); 

1. a consolidation unit which generates final output image data to be displayed by 
consolidating image data generated for each subspace {paragraph [0045]: "At a step 512, 
the outputs of the individuals GPUs are combined by blending. "; Figure 7). 

57. Foran discloses, "image blending accounts for the spatial relationships between the 
viewing position and the rendered images so that objects that are more near the viewing position 
overwrite those that are more distant" {paragraph [0045]). Foran does not expressly disclose the 
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image data has "per-pixel Z values indicating depth information on a pixel by pixel basis for 
each subspace" or consolidating image data "having per-pixel z-values." 

58. With regard to claim 33, Myhrvold et al discloses "an image processing apparatus 
comprising: generating independent image data having per-pixel Z values indicating depth 
information on a pixel by pixel basis for each subspace (lines 37-41 of column 67: "...and Z is 
the Z-value which represents the depth of the pixel from the eye point, Mis a 4x4 pixel coverage 
bitmask for each pixel which is partially covered... "); and image data having per-pixel Z values 
generated for a subspace (lines 12-16 of column 66: "To perform hidden surface removal the 
pixel engine 406 compares depth values for incoming pixels (fully covered pixels or pixel 
fragments) with pixels at corresponding locations in the pixel or fragment buffers"). 

59. At the time of the invention, it would have been obvious to a person of ordinary skill in 
the art to use the per-pixel depth image data of Myhrvold et al in Foran. The motivation for 
doing so would have been to accurately determine visible surfaces, as suggested in Myhrvold et 
al in lines 37-41 of column 67. Therefore, it would have been obvious to combine Myhrvold et al 
with Foran to obtain the invention specified in claim 33. 

Response to Arguments 

60. Applicant's arguments with respect to all previously presented claims have been 
considered but are moot in view of the new ground(s) of rejection necessitated by Applicant's 
amendment. 

Allowable Subject Matter 

61 . Claims 7, 31, 32 and 34 are rejected under 35 U.S.C. 112, first paragraph, for lack of 
enablement, but contain limitations not found in the prior art of record. 
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62. Claim 3 1 depends from claim 1 , and recites the following limitation in combination with 
the limitations recited in the parent claim that is not found in the prior art: the grouping unit 
groups the intersecting three-dimensional objects, to each of which a different rendering strategy 
is applied, into the separate groups 

63. With regard to claim 31, U.S. Patent No. 6,016,150 to Lengyel et al discloses grouping 
geometry and a different rendering strategy is applied to the separate groups (lines 54-59 of 
column 10). U.S. Patent Application Pub. No. 2002/0015055 to Foran discloses three- 
dimensional subspaces. However, one of ordinary skill in the art would not have included a 
grouping unit that groups the intersecting three-dimensional objects, to each of which a different 
rendering strategy is applied, into the separate groups, in combination with the limitations recited 
in claim 1 in Foran. 

64. Claim 32 depends from claim 1, and recites the following limitation in combination with 
the limitations recited in the parent claim that is not found in the prior art: "the rendering 
processing unit generates the independent image data for each subspace, the independent image 
data having per-pixel Z values indicating depth information on a pixel by pixel basis; and the 
consolidation unit generates the final output image data to be displayed by performing Z-merge 
processing of the image data generated for each subspace according to the per-pixel Z values, 
when three-dimensional subspace, each of which contains at least one three-dimensional object 
belonging to the same group, are allowed to spatially overlap one another." 

65. With regard to claims 32 and 7, U.S. Patent No. 5,867,166 to Myhrvold et al discloses 
"per-pixel Z values" (lines 37-41 of column 67) for determining visible surfaces to be rendered to 
an image. U.S. Patent Application Pub. No. 2002/0015055 to Foran discloses "an image 
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processing method comprising dividing a space into three-dimensional subspaces which are 
allowed to spatially overlap one another {paragraph [0040]). While it would have been obvious 
to a person of ordinary skill in the art to use the per-pixel depth image data of Myhrvold et al in 
Foran to accurately determine visible surfaces, it would not have been obvious to "the rendering 
processing unit generates the independent image data for each subspace, the independent image 
data having per-pixel Z values indicating depth information on a pixel by pixel basis; generates 
the final output image data to be displayed by performing Z-merge processing of the image data 
generated for each subspace according to the per-pixel Z values, when three-dimensional 
subspace, each of which contains at least one three-dimensional object belonging to the same 
group, are allowed to spatially overlap one another," as the spatial sub volumes disclosed in 
Foran are ordered according to depth from the viewpoint (paragraph [0045]). However, pixel by 
pixel z values could be used to resolve visibility within the sub-volumes taught by Foran, which 
is excluded by the claim language. Claim 7 depends from claim 32 and therefore contains subject 
matter not found in the prior art. 

66. With regard to claim 34, U.S. Patent No. 5,867,166 to Myhrvold et al discloses "per-pixel 
Z values" {lines 37-41 of column 67) for determine visible surfaces to be rendered to an image. 
U.S. Patent Application Pub. No. 2002/0015055 to Foran discloses "an image processing method 
comprising dividing a space into three-dimensional subspaces which are allowed to spatially 
overlap one another {paragraph [0040]). In the parent claim 34, Foran shows "a consolidation 
unit which generates final output image data to be displayed by consolidating image data 
generated for each subspace {paragraph [0045]: "At a step 512, the outputs of the individuals 
GPUs are combined by blending. "; Figure 7). Myhrvold et al discloses "an image processing 
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apparatus comprising: generating independent image data having per-pixel Z values indicating 
depth information on a pixel by pixel basis for each subspace (lines 37-41 of column 67); and 
image data having per-pixel Z values generated for a subspace (lines 12-16 of column 66). While 

i 

it would have been obvious to a person of ordinary skill in the art to use the per-pixel depth 
image data of Myhrvold et al in Foran, it would not have been obvious to provide a 
"consolidation unit which generates the final output image data to be displayed by performing Z- 
merge processing of the image data generated for each subspace according to the per-pixel Z 
values, when three-dimensional subspaces, each of which contains at least one three-dimensional 
object belonging to the same group, are allowed to spatially overlap." 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's disclosure. 
U.S. Patent No. 5,757,385 Narayanaswami et al discloses a system of rendering using multiple 
processors. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jason M. Repko whose telephone number is 571-272-8624. The 
examiner can normally be reached on Monday through Friday 8:30 am -5:00 pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Ulka Chauhan can be reached on 571-272-7782. The fax phone number for the 
organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). If you would 
like assistance from a USPTO Customer Service Representative or access to the automated 
information system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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